Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an edit deployment API #2515

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add an edit deployment API #2515

wants to merge 1 commit into from

Conversation

jackkleeman
Copy link
Contributor

Adds an alternative to force which allows you to update an existing deployment ID to have a new arn/uri/etc, including adding services and handlers if necessary. For break glass operations to get you out of a jam with failing invocations.

We do allow this operation to move existing deployments to point to the same URI as other deployments, an invariant we previously kept. However, in this case, we will not allow further force deploys against that URI until some deployments are removed to make it unambigious which deployment ID the force should subsume

The recommended workflow is as follows:

  1. Notice failing invocations that can only be resolved with a code change
  2. PUT to the deployment ID to change to a new URI/ARN with the code change
  3. If, mistakenly, you have POSTed that new change and updated latest to point to your change, you will now want to move the failing invocations on the old deployment id to that new ARN. You may PUT the same ARN again to the old deployment id. However, you should then delete the old deployment ID when the invocations have completed. In the meantime, force deploys against that ARN won't be possible.

If you notice failing invocations after you've already deployed some other change, because the previous deployment hasn't drained, the workflow is as follows:

  1. Craft the fix, and cherry pick it onto the version backing deployment ID that the invocations are failing on. PUT to that deployment ID with the cherry picked code
  2. If that resolved the issue, now rebase your fix on whatever backs your latest deployment, and PUT to that deployment with that code.

Copy link

github-actions bot commented Jan 20, 2025

Test Results

  7 files  ±0    7 suites  ±0   4m 7s ⏱️ +42s
 47 tests +2   46 ✅ +2  1 💤 ±0  0 ❌ ±0 
182 runs  +8  179 ✅ +8  3 💤 ±0  0 ❌ ±0 

Results for commit bed347c. ± Comparison against base commit 0ac0d58.

♻️ This comment has been updated with latest results.

Copy link
Contributor

@slinkydeveloper slinkydeveloper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes, and then lgtm

existing: DeploymentId,
},
#[error("an update deployment operation must provide an endpoint with the same services and handlers. The update tried to remove the services {0:?}")]
#[code(restate_errors::META0006)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would define a new error code for errors related to update. META0006 seems wrong as it talks about adding deployments. In the error code you can describe in a more lengthy manner what to do

Comment on lines +260 to +265
pub fn update_deployment(
&mut self,
deployment_id: DeploymentId,
deployment_metadata: DeploymentMetadata,
services: Vec<endpoint_manifest::Service>,
) -> Result<(), SchemaError> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's me, but I couldn't find a check that verifies the supported service protocol range is the same. This is definitely the most important thing to check, as there is a super high change that if they don't match, the invocation will get broken/corrupted.

Comment on lines +301 to +313
let handlers = DiscoveredHandlerMetadata::compute_handlers(
service
.handlers
.into_iter()
.map(|h| {
DiscoveredHandlerMetadata::from_schema(
service_name.as_ref(),
service_type,
h,
)
})
.collect::<Result<Vec<_>, _>>()?,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're missing somewhere to preserve the private handler flag? Maybe add a unit test to check that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants